Scalable video format conversion system

ABSTRACT

The present invention discloses a scalable video format conversion system for utilizing various system resources to provide a progressive video signal. The scalable video format conversion system has a scalable motion-adaptive de-interlacing system for converting an interlaced video signal into the progressive video signal according to a motion situation of an image area. The scalable motion-adaptive de-interlacing system includes a motion detection apparatus detecting the motion situation of the image area according to the availability of the various system resources or the status of the scalable video format conversion system.

CROSS REFERENCE TO RELATED APPLICATIONS

This continuation application claims the benefit of U.S. patentapplication Ser. No. 10/710,871, filed on Aug. 9, 2004 and includedherein by reference.

BACKGROUND

The invention relates to a scalable video format conversion system, andmore particularly, to a system which dynamically determines how manyvideo fields are used in motion detection process when converting aninterlaced video signal into a progressive video signal.

Generally speaking, there are two kinds of video signal, one isprogressive video signal, and the other is interlaced video signal. In aprogressive video signal, a video frame comprises all points (or pixels)of the image at any given sampling time. In an interlaced video signal,a video field comprises only half the pixels of the image at any givensampling time, and another video field at the next sampling timecomprises the remaining half of the pixels. The advantage of interlacedscanning is that a high field rate can be achieved. However, when aninterlaced video signal must be played on a progressive display device(such as a computer monitor), the interlaced video signal must beconverted into a progressive video signal through “interlaced toprogressive conversion”.

Conventional systems use a dedicated video format conversion chip toprocess video format conversion. The calculation requirements, thememory volume requirements, and the memory bandwidth requirements areroughly fixed, hence it does not require complex system resourcesmanagement. With the advances of modern computer systems, some chipswith integrated video processing capabilities are proposed. Systemresources management therefore becomes more and more complicated. Asimple solution is to sufficiently design the system for handling thestrictest situation. However, systems sufficient for handling thestrictest situation cost much, and the strictest situation is not themost regular case. The drawback is that the system resources will not beused efficiently.

In U.S. Pat. No. 6,452,639 disclosed by Wagner, a de-interlacing methodfor dynamically determining which interpolation algorithm being usedaccording to the state of system resources is proposed. With the method,when system resources are plentiful, the system uses a more complexinterpolation algorithm, and when system resources are sparse, thesystem uses a simpler interpolation algorithm.

However, using the motion-adaptive de-interlacing method is the mostefficient way to process interlaced to progressive conversion. Themotion-adaptive de-interlacing method generally includes two steps. Thefirst step involves processing motion detection, which means detecting amotion situation by checking a fix number of video fields of theinterlaced video signal. Then, the second step involves selecting aproper interpolation algorithm according to the detected motionsituation.

FIG. 1 shows a conventional video format conversion system. In FIG. 1,the video format conversion system 100 is a single chip containing anMPEG codec circuit 120 and a video format conversion circuit 140. Thevideo format conversion system 100 utilizes the variable systemresources 180 (including memory bandwidth, memory capacity, etc.) toprocess MPEG encoding/decoding and video format conversion. The videoformat converter 140 includes a motion detector 150 and an interpolator160. The motion detector 150 determines a motion situation by checking afix number of video fields in an interlaced video signal. Theinterpolator 160 converts the interlaced video signal into a progressivevideo signal with a chosen interpolation algorithm, which is chosenaccording to the motion situation.

In motion detection process, in actuality, the number of utilized videofields does not have to be fixed. Generally, the number of utilizedvideo fields ranges from 2 to 6 fields (or even more than 6 fields). Themore fields are utilized, the more accurate the detected motionsituation will be. However, more system resources will be consumed whenmore video fields are utilized. FIG. 2 shows an example of 3-fieldmotion detection. For determining the motion situation of aninterpolating point X in field T, the 3-field motion detection finds thedifference between the pixel value of point A in field T−1 and the pixelvalue of point B in field T+1, then uses a threshold to distinguish theresult between static and dynamic. When |A−B|>threshold, the result isdynamic; when |A−B|<threshold, the result is static. The system canselect a proper interpolation algorithm according to whether the resultis dynamic or static.

FIG. 3 shows an example of 6-field motion detection. The point X infield T is the interpolated point. When |A−B|>threshold or|C−D|>threshold or |E−F|>threshold or |C−G|>threshold or |E−H|>thresholdor |A−I|>threshold, the result is dynamic, otherwise the result isstatic. The situation shown in FIG. 3 is only an example. The usedfields for motion detection process can be from T−3 to T+2, from T−2 toT+3, or other successive 6 fields.

As mentioned above, when more video fields are utilized in motiondetection process, the more system resources will be consumed. However,in the prior art as shown in FIG. 1, conventional video formatconversion system does not determine the number of video fields used inmotion detection process dynamically according to the availability ofvariable system resources. That is, the system does not always use thebest motion detection method acceptable to the available systemresources at any given time. The draw back of the above mentionsituation is that the system resources are not used efficiently.

SUMMARY

It is therefore an objective of the present invention to provide ascalable video format conversion system that can dynamically determinethe number of video fields used in motion detection process according tothe situation of system resources to solve the above-mentioned problems.

According to an embodiment of the present invention, a scalable videoformat conversion system is provided for utilizing various systemresources to provide a progressive video signal. The scalable videoformat conversion system has a scalable motion-adaptive de-interlacingsystem for converting an interlaced video signal into the progressivevideo signal according to a motion situation of an image area. Thescalable motion-adaptive de-interlacing system includes a motiondetection apparatus detecting the motion situation of the image areaaccording to the availability of the various system resources or thestatus of the scalable video format conversion system.

It is an advantage of the present invention that the disclosedvariable-field motion detection apparatus dynamically selects a properdetection number according to the availability of system resources andthe status of the scalable video format conversion system. Hence thesystem resources will be used more efficiently, and better video effectscan be achieved.

These and other objectives of the present invention will no doubt becomeobvious to those of ordinary skill in the art after reading thefollowing detailed description of the preferred embodiment that isillustrated in the various figures and drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a block diagram of a conventional video format conversionsystem.

FIG. 2 shows an example illustrating 3-field motion detection.

FIG. 3 shows an example illustrating 6-field motion detection.

FIG. 4 shows an embodiment of a scalable video format conversion systemaccording to the present invention.

FIG. 5 shows a block diagram of the scalable motion-adaptivede-interlacing system of FIG. 4.

FIG. 6 shows a lookup table for determining the detection numberaccording to the available memory bandwidth.

FIG. 7 shows a lookup table for determining the detection numberaccording to the user-selectable operation mode.

FIG. 8 shows a second embodiment of a variable-field motion detectionapparatus according to the present invention.

FIG. 9 shows a third embodiment of a variable-field motion detectionapparatus according to the present invention.

DETAILED DESCRIPTION

FIG. 4 shows an embodiment of a scalable video format conversion systemaccording to the present invention. In this embodiment, the scalablevideo format conversion system 400 is implemented in a single chip 400having both MPEG encoding/decoding capability and video formatconversion capability. The variable system resources 480 (such asmemory, memory bandwidth, etc.) are shared by an MPEG codec circuit 420and a scalable motion-adaptive de-interlacing system 440 of the singlechip 400. When converting an interlaced video signal into a progressivevideo signal, the scalable motion-adaptive de-interlacing system 440accesses a plurality of video fields to determine a motion situation ofan image area. The number of the plurality of video fields accessed bythe scalable motion-adaptive de-interlacing system 440 is equal to adetection number, and the detection number is dynamically determined bya mode control module 410 according to the availability of the systemresources 480 and/or the status of the scalable video format conversionsystem 400. In the scalable motion-adaptive de-interlacing system 440, avariable-field motion detection apparatus 450 handles the motiondetection process. After the motion situation is determined, aninterpolator 460 chooses a proper interpolating algorithm to convert theinterlaced video signal into the progressive video signal.

FIG. 5 shows a block diagram of the scalable motion-adaptivede-interlacing system 440. In the embodiment shown in FIG. 5, thevariable-field motion detection apparatus 450 includes five motiondetectors 455 a-e. Each of the motion detectors 455 a-e implementsmotion detection process with a specific number of video fields. Thedetection number determined by the mode control module 410 will decidewhich of the motion detectors 455 a-e will be used. For example, whenthe detection number determined by the mode control module 410 is six,the motion detector 455 e will be used to implement motion detectionprocess. When the detection number is four, the motion detector 455 cwill be used to implement motion detection process. As mention above,the interpolator 460 selects a proper interpolation algorithm accordingto the motion situation detected by the variable-field motion detectionapparatus 450. Please note that, in this embodiment, each motiondetector 455 a-e handles motion detection process with a specific numberof video fields. However, it is also possible that the variable-fieldmotion detection apparatus 450 handles motion detection process withdifferent numbers of video fields by itself, rather than containing fivemotion detectors. Such apparatus will be discussed in more detail later.

As mentioned above, the mode control module 410 determines the detectionnumber according to the availability of the system resources 480 and/orthe status of the scalable video format conversion system 400. Forexample, the availability of the system resources 480 can be determinedaccording to the computational power of the scalable video formatconversion system 400, the available space in a memory (not shown) ofthe system resources 480, the available memory bandwidth, or thelimitation of power consumption, etc. When the availability of thevariable system resources 480 shows that there are abundant availablesystem resources, the mode control module 410 will set the detectionnumber larger; when the availability of the variable system resources480 shows that there are only sparse available system resources, themode control module 410 will set the detection number smaller. Takingmemory bandwidth as example, the table shown in FIG. 6 is a lookup tablefor determining the detection number according to the available memorybandwidth. It is noted that the lookup table shown in FIG. 6 is onlyillustrative and not limiting.

Further, the status of the scalable video format conversion system 400can be determined according to the bit rate of a video bit-stream or anaudio bit-stream, the data processing rate of a video codec or an audiocodec, the work load of a parser, the display or decoding load of asub-picture controller, the display workload of an on-screen-displaycontroller (OSD), or the user-selectable operation mode, etc. When thestatus of the scalable video format conversion system 400 shows thatthere are still abundant system resources for the scalablemotion-adaptive de-interlacing system 440 to use, the mode controlmodule 410 could set the detection number larger. When the status of thescalable video format conversion system 400 shows that there are onlysparse system resources left for the scalable motion-adaptivede-interlacing system 440 to use, the mode control module 410 could setthe detection number smaller.

The above-mentioned “user-selectable operation mode” could be aLetterbox mode, Pan-scan mode, PAL-to-NTSC conversion mode, NTSC-to-PALconversion mode, zoom in mode or zoom out mode, etc. Each specificoperation mode could corresponds to a specific detection number. It isalso noted that the modes shown in FIG. 7 are only illustrative and notlimiting.

In addition to FIG. 4, the mode control module 410 can also beimplemented inside a memory controller (not shown in FIG. 4) of thescalable video format conversion system 400, to dynamically adjust thedetection number according to the bandwidth workload of the memorycontroller. When the memory controller has a smaller bandwidth workload,the mode control module could set the detection number larger. When thememory controller has a larger bandwidth workload, the mode controlmodule could set the detection number smaller.

As mentioned above, rather than containing five motion detectors asshown in the embodiment of FIG. 5, the variable-field motion detectionapparatus 450 could also handle motion detection process with differentnumbers of video fields by itself. FIG. 8 shows a second embodiment of avariable-field motion detection apparatus 800 according to the presentinvention. The variable-field motion detection apparatus 800 processesmotion detection process by accessing a plurality of video fields of aninterlaced video signal to determine the motion situation of an imagearea. In this embodiment, the variable-field motion detection apparatus800 comprises six pixel difference circuits 810 a-f, a decision circuit890, and six multiplexers 850 a-f. As a whole, the multiplexers 850 a-fcould be regarded as a field-number adjuster. Each one of the pixeldifference circuits 810 a-f computes the pixel value difference betweena point on two different video fields and generates a detection value asa result. Referencing the example shown in FIG. 3, in this embodimentthe inputs of the pixel difference circuits 810 a-f are pixel values ofpoints A, B, C, D, E, F, G, H, I shown in FIG. 3. Each of the pixeldifference circuits 810 a-f contains a subtracter 820 a-f and anabsolute value circuit 830 a-f, which can be used to compute theabsolute value of the difference between two pixel values. After adetection value of a pixel difference circuit passes through acorresponding multiplexer, a corresponding comparator will compare thedetection value with a predetermined threshold, then generates a booleanvalue as a result. Please note that the predetermined thresholds used bythe comparators 860 a-f could have a common value or have differentvalues. A logic OR operation is then preformed on these boolean valuesBLa-f to generate the motion detection result. In the above-mentionedsituation, the variable-field motion detection apparatus 800 can beregarded as a 6-field motion detector.

However, 6-field motion detection is not necessary at all times. Hencethe field-number adjuster can dynamically adjust the number of videofields used in motion detection process. For example, when 5-fieldmotion detection is applied by the system, the value inputted to thecomparator 860 f will be set to ‘0’ by the multiplexer 850 f (which ispart of the field-number adjuster). In this way, the T−3 field in FIG. 3will have no influence on the motion detection result. When 4-fieldmotion detection is applied by the system, the values inputted tocomparators 860 f, 860 e, and 860 d will be set to ‘0’ by themultiplexers 850 f, 850 e, and 850 d (which are all parts of thefield-number adjuster). In this way, the T−3 and T+2 fields in FIG. 3will have no influence on the motion detection result. Hence, it can beseen that by switching the multiplexers 850 a-e properly, thevariable-field motion detection apparatus 800 could adjusts the numberof video fields used in motion detection process dynamically. Pleasenote that, setting the values inputted to the comparators 860 a-e to ‘0’by properly switching the multiplexers 850 a-f is just an example. Toadjust the number of video fields used in motion detection process, eachvalue inputted to a comparator in FIG. 8 could also be set to any valuesmaller the threshold used by a corresponding comparator.

In addition, rather than being located between the pixel differencecircuits 810 a-f and the comparators 860 a-f, the field-number adjuster(which includes the multiplexers 850 a-f in FIG. 8 in this embodiment)could also be located between the comparators 860 a-f and the logic ORcircuit 870 to set some of the boolean values BLa-f to zero; or belocated in front of the pixel difference circuits 810 a-f to set someoutput end pairs to the same value. Above are some possibleimplementations of the field-number adjuster.

Sometimes motion detection with few number of video fields used will notbe able to detect fast moving objects. In this situation the edges of afast moving object might appear as ragged sawtooths rather than smoothcurves. To solve this problem, the variable-field motion detectionapparatus 800 of the present invention can also operate in conjunctionwith a mouse teeth detector (also called a sawtooth detector), as shownin FIG. 8, to prevent ragged sawtooths from appearing. For more detailson sawtooth detectors, please refer to U.S. Pat. No. 5,625,421.

FIG. 9 shows a third embodiment of a variable-field motion detectionapparatus 900 according to the present invention. The main differencebetween the variable-field motion detection apparatuses 900 and 800 isthat the select module 960 of decision circuit 990 selects a largestdetection value output by the pixel difference circuits 910 a-f. Thecomparator 970 can then compare the largest detection value with apredetermined threshold to get the final motion detection result. Themultiplexers 950 a-f, which could be regarded as a field adjuster, setsome detection values output by the pixel difference circuits 910 a-f tozero (or another value smaller than the predetermined threshold). Hencedynamically adjusting the used field number can be achieved. Please notethat, rather than being located between the pixel difference circuits910 a-f and the select module 960, the field-number adjuster (whichincludes multiplexers 950 a-f in this embodiment) could also be locatedin front of the pixel difference circuits 910 a-f to set some input endpairs of some pixel difference circuits to the same value. For example,when the detection number is six, the field-number adjuster could passpixel values on six different video fields to the inputs of the pixeldifference circuits 910 a-f. When the detection number is five, thefield-number adjuster could set the two values inputted to the pixeldifference circuits 910 f as the same value, at this time only fivevideo fields will affect the motion detection result. When the detectionnumber is four, the field-number adjuster could set the two valuesinputted to the pixel difference circuits 910 f as the same value, setthe two values inputted to the pixel difference circuits 910 e as thesame value, and set the two values inputted to the pixel differencecircuits 910 d as the same value, at this time only four video fieldswill affect the motion detection result. Its apparent that throughswitching the field-number adjuster properly, the number of video fieldsused in motion detection process could be adjusted dynamically,according to the detection number determined by the mode control module410. Furthermore, the variable-field motion detection apparatus 900 ofthis embodiment can also operate in conjunction with a mouse teethdetector as mentioned before.

Those skilled in the art will readily observe that numerousmodifications and alterations of the device and method may be made whileretaining the teachings of the invention. Accordingly, the abovedisclosure should be construed as limited only by the metes and boundsof the appended claims.

1. A scalable video format conversion system for utilizing varioussystem resources to provide a progressive video signal, the scalablevideo format conversion system comprising: a scalable motion-adaptivede-interlacing system for converting an interlaced video signal into theprogressive video signal according to a motion situation of an imagearea, the scalable motion-adaptive de-interlacing system comprising: amotion detection apparatus performing a motion detection process uponthe image area for detecting the motion situation of the image area,wherein the motion detection process performed upon the image area isconfigured according to the availability of the various system resourcesor the status of the scalable video format conversion system.
 2. Thescalable video format conversion system of claim 1, wherein the scalablemotion-adaptive de-interlacing system further comprises a mode controlmodule generating a mode control signal to the motion detectionapparatus according to the availability of the various system resourcesor the status of the scalable video format conversion system.
 3. Thescalable video format conversion system of claim 2, wherein the motiondetection apparatus processes a plurality of video fields for thedetection of the motion situation of the image wherein the number of theplurality of processed video fields corresponds to the mode controlsignal.
 4. The scalable video format conversion system of claim 3,wherein the motion detection apparatus comprises: a plurality of pixeldifference circuits, wherein each pixel difference circuit computes adetection value according to the difference between the pixel values ofa single pixel in two different video fields; a decision circuit coupledto the pixel difference circuits, determining the motion situation ofthe image area according to the detection values; and a field-numberadjuster adjusting the pixel difference circuits or the decision circuitto eliminate the effect of some video fields according to the modecontrol signal.
 5. The scalable video format conversion system of claim4, wherein each pixel difference circuit comprises: a subtractercomputing the difference between the pixel values of a single pixel intwo different video fields; and an absolute circuit coupled to thesubtracter, computing the absolute value of the output signal of thesubtracter as the detection value.
 6. The scalable video formatconversion system of claim 4, wherein the decision circuit comparingeach detection value with a related threshold to accordingly generate aboolean value, and then using the boolean values to compute the motionsituation of the image area.
 7. The scalable video format conversionsystem of claim 6, wherein the field number adjuster sets some of thedetection values to be smaller than the related thresholds according tothe mode control signal.
 8. The scalable video format conversion systemof claim 6, wherein the field number adjuster is for setting some of theboolean values to be zero according to the mode control signal.
 9. Thescalable video format conversion system of claim 4, wherein the decisioncircuit generates a maximum detection value according to the detectionvalues, and comparing the maximum detection value with a threshold todetermine the motion situation of the image area.
 10. The scalable videoformat conversion system of claim 9 wherein the field number adjustersets some of the detection values to be smaller than the thresholdaccording to the mode control signal.
 11. The scalable video formatconversion system of claim 2, wherein the motion detection apparatusprocesses a plurality of video fields for the detection of the motionsituation of the image wherein a number of the plurality of processedvideo fields is adjustable in accordance with the mode control signal.12. The scalable video format conversion system of claim 11, wherein thenumber of the plurality of processed video fields ranges from 2 to 6.13. The scalable video format conversion system of claim 1, wherein thescalable motion-adaptive de-interlacing system comprises at least onemotion detector and at least one interpolator controlled by the motiondetector.
 14. The scalable video format conversion system of claim 1,wherein the availability of the various system resources is dependentupon the available computational power, the available memory space, theavailable memory bandwidth, or the system power of the scalable videoformat conversion system.
 15. The scalable video format conversionsystem of claim 1, wherein the status of the scalable video formatconversion system is dependent upon the bit rate of a video bit-streamor an audio bit-stream.
 16. The scalable video format conversion systemof claim 1, wherein the status of the scalable video format conversionsystem is dependent upon the data processing rate of a video decoder, anaudio decoder, a video encoder or an audio encoder.
 17. The scalablevideo format conversion system of claim 1, wherein the status of thescalable video format conversion system is dependent upon the workloadof a decoder parser or an encoder parser.
 18. The scalable video formatconversion system of claim 1, wherein the status of the scalable videoformat conversion system is dependent upon the display or decodingworkload of a sub-picture controller.
 19. The scalable video formatconversion system of claim 1, wherein the status of the scalable videoformat conversion system is dependent upon the display workload of anon-screen-display (OSD) controller.
 20. The scalable video formatconversion system of claim 1, wherein the status of the scalable videoformat conversion system is dependent upon the memory bandwidth workloadof a memory controller.
 21. The scalable video format conversion systemof claim 1, wherein the status of the scalable video format conversionsystem is dependent upon a user-selectable display mode, such asLetterbox, Pan-scan, NTSC-to-PAL conversion, PAL-to-NTSC conversion,Zoom in, or Zoom out.
 22. A scalable video format conversion system forutilizing various system resources to provide a progressive videosignal, the scalable video format conversion system comprising: ascalable motion-adaptive de-interlacing system for converting aninterlaced video signal into the progressive video signal according to amotion situation of an image area, the scalable motion-adaptivede-interlacing system comprising: a motion detection apparatus detectingthe motion situation of the image area according to the availability ofthe various system resources or the status of the scalable video formatconversion system; and a mode control module generating a mode controlsignal to the motion detection apparatus according to the availabilityof the various system resources or the status of the scalable videoformat conversion system.